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ABSTRACT 

This paper reviews the development of a curriculum 
assessment plan by the Business and Management Division of Cardinal 
Stritch College in Milwaukee, Wisconsin, and reports the results of a 
study to determine the effectiveness of the plan. The division, which 
delivers accelerated instruction in evening courses, used the 
comprehensive outcomes assessment program (COAP) and the core 
curriculum assessment program (CCAP) to evaluate the division's 
programs and student achievement. The division used the COAP and CCAP 
measures to compare the pre" and post“test scores of 78 students who 
have completed the Bachelor of Science in Business Management program 
and 41 students who have completed the Master of Science in 
Management program. It also used the measures to assess the pre“test 
scores of 67 students who did not complete these programs. The study 
found that students who completed the Bachelor's or Master's programs 
exhibited a significant increase in business management skills. It 
also found l"hat students who did not complete the Bachelor'^s program 
had signi f icant ly higher pre-test scores than students who received 
their degree, while students who did not complete the Master's 
program had lower pre-test scores than those who did. (MDM) 
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Abstract 

The Core Curriculum Assessment Program (CCAP) was dev eloped by the 
American College Testing (ACT) program for the American Assembly of Collegiate 
Schools of Business (AACSB). The test evaluates student achievement and curriculum 
standards in seven disciplinary areas within the business environment. The test bank of 
more than 800 questions was used as a major component in a comprehensive outcomes 
assessment program for the business and management division. This paper will address 
the assessment process, development, methodology, and results of the CCAP for a 
nontraditional business program. Pre- and post-test scores are compared between groups, 
as well as test scores of individual students, to analvze the validitv of the CCAP. 
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Overview 

Cardinal Stritch College is an independent Catholic institution of post-secoridar\' 
education, sponsored by the Sisters of St. Francis of Assisi. The College, rooted in the 
liberal arts, provides graduate and undergraduate programs to prepare students for life and 
for professional careers. The College assists women and men in pursuing lifelong 
learning. It provides both traditional and non-traditional approaches to meet the 
educational needs of a diverse student body. There are three main divisions within the 
institution: Liberal Arts. Teacher LJucation. and Business and Management Division. 

The College had been involved in assessment and cyclical-self assessment since 1991 but 
its nontraditional programs presented several unique circumstances to be addressed. 

The Business and Management Division of the College consists of si.x different 
majors for the working-adult students at the associate, bachelor, and graduate level. 
Classes are held one night a week from 6:00 pm to 10:00 pm and are delivered in an 
accelerated format. Enrollment is limited to tw ny-two students per class and are started 
approximately every' month depending on the number of individuals registered. There are 
no traditional semester starts and while this accelerates all aspects of the program 
delivery it also creates various advantages. These are lock-step programs where students 
enter and complete courses in specified sequences w ith the same cluster of students. 

Cardinal Stritch College's Assessment Plan was developed through the efforts of 
many members of the College community. Committed to College-wide institutional 
effectiveness, every effort was made to provide information to students, faculty, and staff 
as to the importance of developing such a plan. 
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The following principles guided the College in developing the plan: 

1 . that the mission of the College be reflected in the formulation of the plan 
and the utilization of results; 

2. that the plan be comprehensive in focus to include the entire institution; 

3. that the plan have institutional support by hiring a Director of Institutional 
Research. This individual would be responsible for coordinating the 
assessment activities and serv’e as a member of the Assessment 
Committee; 

4. that the plan be cost-effective and utilize already established measures of 
assessment; 

5. that communication systems utilizing current College structures be 
established by which students receive information regarding the 
assessment initiatives and their purpose, and that College-wide 
presentations be made on the plans for assessing institutional 
effectiveness; 

6. that multiple measures be selected through consultation and 
recommendation of representatives from the faculty and administration: 

7. that a faculty committee including professional staff members from the 
academic area be established to review results, analyze data, refine 
assessment processes and recommend change; 

8. that systems be designed to provide feedback on results to the College 
constituencies; 

As it relates to Assessment of Student Achievement: 

9. that students be assessed at multiple points during their educational 
experience; 

1 0. that the assessment of student achievement lead to the improvement of the 
educational experience of CSC students; 

1 1 . that the plan employ a variety of methods of measurement of student 
achievement, including both quantitative and qualitative measures; 

12. that the plan be linked to the College educational experience outcomes, 
general educational outcomes, academic program outcomes and individual 
course outcomes; 

1 3. that faculty review results and make decisions regarding program 
improvement. 



The principles that guided the formulation of Cardinal Stritch College's Plan for 
Institutional Assessment have been realized. 
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The Mission of Cardinal Stritch College states that it: 

prepares students for life and for professional careers; 
meets the educational needs of a diverse student body; 
addresses the emerging needs of students; 

provides direct service to people who might not otherwise be served; and 
fosters the moral, spiritual, social and physical development of each 
person. 

Based on these mission statements, the institution designed an assessment process that 
examined the College against its mission and institutional goals. 

The Comprehensive Outcomes Assessment Program (CO.-XP) was designed to 
provide a system of continuous evaluation of the Business and Management Division of 
the College. A multiplicity of assessment instruments were either purchased, developed, 
or devised 'o review the cognitive areas of the various different majors, as well as the 
liberal arts component, and affective elements. Pre- and post-testing was an important 
aspect of COAP. Both surveys and cognitive testing were utilized, as well as norm- 
referenced and criterion-referenced testing. 

Three main points were considered essential within the development process. The 
assessment instruments had to address the mission of the College, they had to be 
comprehensive, and the proeess was to be well-established so it would operate in a Total 
Quality Management mode, applying concepts of control, quality process, and customers 
to management, e.g. continuous improvement. 

The Core Curriculum .Assessment Program (CCAP) was de\eloped by the 
American Assembly of Collegiate Schools of Business Programs (AACSBP) to e\ aluate 
student achie\ement in se\en disciplinar\- areas within the business environment. For the 
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major field of study in the nontraditional Business and Management Division, the 
AACSBP CCAP was selected because it was cost-effective, it pro\ ided a great deal of 
flexibility— it was adjusted for the six different majors in the division, and it was criterion- 
referenced. The test bank of more than 800 questions was used as a major component in 
a comprehensive assessment program for a business program. The assessment process, 
development, methodology, and results of the CCAP for a nontraditional program 
provided the basis for the research in this paper. 

The following table outlines the Comprehensive Outcomes Assessment Program: 



Associate Bachelor 
Pre-assessment Night 

New Student Reply Form 
Attitudinal Survey 
ACT ASSET Test; B 
AACSB CCAP (90 min) 
Major Field Assessment 



Associate. Bachelor 
Post-assessment Niaht 

Student Opinion Survey 
Attitudinal Survey 
ACT ASSET Test: Cl 
AACSB CCAP (90 min) 
Major Field Assessment 



Graduate Le\ el 
Pre-assessment Night 

New Student Reply Form 
Attitudinal Survey 
AACSB CCAP (90 min) 
Major Field Assessment 



Graduate Level 
Post-assessment Nistht 

Student Opinion Surves 
Attitudinal Survey 
AACSB CCAP (90 min) 
Major Field Assessment 



fhe information collected was used for assessment purposes and program 
evaluation only. All data were held in confidence and only group data are reported. 
Completion of the assessment program is required of all students but there are no pass or 
fail "grades" distributed. In other words, students must only complete the assigned 
instruments to graduate from tiie college, they do not have to attain a certain lev el, or 
score, to graduate. 

1 he Comprehensive Outcomes Assessment Plan was organized so that the results 
were used to enhance the quality of the program, while the process itself was reviewed 
and updated to increase the efficiency and effectiveness of the institution. The 
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assessment plan was implemented into a purposeful program that was institution-wide, 
while the assessment helped shape institutional priorities. 

Goal of the study 

The purpose of this study is two-fold. One purpose is to identify differences 
between pre-test and post-test scores of students who have completed the curriculum. 

The second purpose is to see if there are differences betw een pre-test scores of those 
completing the program and those not completing the program. For purposes of this 
paper, these groups will be called "Survivors" and “Non-survivors." 

After considerable research, the analysis focused on students in two business 
programs; undergraduate students in the Bachelor of Science in Management (BSM) and 
graduate students in the Master of Science in Management (MSM) programs. 

The first and last night of the programs are reserv ed for outcomes assessment 
acti\ ities. including administering the CCAP. As a result, pre-and post-test scores may 
be matched to individuals who completed (Survivors) as well as defining those who did 
not complete (Non-sur\ i\ ors). A series of matched pairs t-tests was run as w ere 
independent sample t-tests. 

Background 

This study had its origin in an initial comparison of pre-test with post-test scores 
of one of the programs. Because data had been gathered in groups of students with the 
initiation of the assessment program, groups of pre-test data and post-test data were 
generated as students started and completed the programs. As expected, post-test scores 
were, in the aggregate, higher than pre-test scores. Two concerns immediately presented 
themseh es. The first concern was that the initial observations were not of matched pairs. 




8 



Survivors and Non-survivors 

8 



that is the pre- and post-test pairs did not usually involve the same individuals, or in most 
cases even the same groups of individuals. This made it impossible to peiform dependent 
sample (or matched pairs) t-tests for the effectiveness of the curriculum until very 
recently. The second issue was whether or not improvement in pre- and post-test scores 
was based on improvement in skills due to learning from the curriculum, or whether the 
improvement in scores was due to another factor. Specifically, at issue was whether 
Surv ivors entered the program with a higher level of innate ability as tested by the 
■'^•--assasshient program than Non-survivors. This second question could be phrased "did 
Surv iv ors come from the higher end of the curve in the first place, and is that why the 
post-test results were, in the aggregate, higher than pre-test results?" Put another way. 
one might ask "Is there as selection process going on which is based on something other 
than cognitive ability?" 

Results 

The first question was answered by a series of matched pair t-tests. Table 1 
indicates that the mean differences between pre- and post-test scores were significant at 
the 0.000 level for BSM students and at the 0.019 level for MSM students. 

These results suggest that, for these groups of students, significant increases in 
skills as measured by the CCAP were observed. 
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Table 1 : Matched pair comparison of pre- and post test results for BSM and MSM students 



BSM students 




Number of 






S.E. of 






Variable 


Pairs 


Mean 


SD 


Mean 


t-value 


siu. p. 


Pre-test score 




33.64 


6.39 


0.659 






Post-test score 




37.80 


5.82 


0.724 






Paired Differences 


78 


4.16 


5.80 


0.657 


6.62 


0.000 



MSM students 




Number of 






S.E. of 






Variable 


Pairs 


Mean 




Mean 


t-value 


siiz. p. 


Pre-test score 




37.41 


5.47 


0.877 






Post-test score 




40.07 


5.61 


0.854 






Paired Differences 


41 


2.66 


6.96 


1.087 


2.45 


0.019 



The second part of this study was the more difficult to analyze. The data took 
considerable evaluation to determine which students were not included in the groups of 
Survivors or Non-survivors. Specifically, it was relatively straightforward to determine 
which groups of students had taken the pre-test, but determining which indi\ idual 
students had not taken the post-test and for what particular reasons was more difficult. 
More to the point, determining which students had completed the program but had not 
taken the post-test, albeit perhaps for perfectly legitimate reasons, was a labor-intensive 
task. 

Table 2 indicates that the difference between pre-test scores for Survivors and 
comparable Non-survi\ors for Bachelor level students is significant at the 0.064 le\ el. 
N'on-sur\ ivors had a significantly higher mean jire-test score than Sur\ i\ors. and the 
dispersion for Non-survivors was greater. Table 2 further shows that for Masters le\el 
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students in the MSM program the opposite is the case: Surv'ivors had a higher pre-test 
score than Non-survivors, significant at the 0.070 level. 



Table 2. Independent sample comparison of Pre-test scores for Survivors and Non- 
survivors of BSM and MSM Students 



BSM students 




Number 










Population 


of Cases 


Mean 


SD 


t- value 


siu. p. 


Survivors 


78 


33.6 


6.39 






Non-survivors 


48 


36.1 


8.01 














-1.87 


0.064 






MSM students 








Number 










Population 


of Cases 


Mean 




l-value 


siu. p. 


Survivors 


41 


37.41 


5.61 






Non-survivors 


19 


34.42 


6.30 







1.85 0.070 



This suggests that, for Bachelor-level students, there are measurable differences 
between Survivors and Non-survi\ ors in skills when entering the program. The fact that 
Non-sun ivors have a higher mean test score than Survivors suggests the possibility that, 
at this level, as much as mastering cognitive skills students are going through a selection 
process. In other words, not only are they being taught, they are being sorted on the basis 
of the ability to stay in the program, perhaps on the basis of perseverance. As William 
Disbro wrote. "One of the first things a college degree says about you is that you can 
survive." 

Masters le\ el students present a ver\ different picture. MSM Survivors are 
entering the program v\ith somewhat higher skills than Non-Sun i\ors. Arguabh . the 
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Survivors are being drawn from the high end of the curve, and are selecting themselves 
on the basis of enhancing a body of knowledge, rather than teaching new skills.* 

What are we to make of these conflicting results? These results imply that at the 
undergraduate level considerable growth is achieved by students who are willing to stick 
it out. Since undergraduate Non-survivors have a higher mean than Survivors, it appears 
that this is a test of perseverance more than academics. By virtue of their higher pre-test 
score, we know that the Non-survivors have the cognitive ability to complete this 
program. Graduate school, by its more demanding nature, may require more cognitive 
skill to begin with. Thus, the higher pre-test scores for Survivors indicate that sorting out 
of students is based on academics. The writers will admit that this result w as unexpected. 
In fact, these preliminaiy findings have caused some excitement, and provided fertile 
ground for more study as more data are sifted through. Attention may also turn to a study 
of wh ther undergraduate Non-survivors go on to continue their education at a school that 
they may find more challenging. In short, as is so often the case, this study has raised 
more questions than it has answered. 

However, one of the main goals of the assessment program was to evaluate the 
curriculum and students in the Business and Management Division. One of the results of 
this study was to see the CCAP in a different light. The CCAP has provided more 
infomiation regarding the academic effectiveness of the program which should lead to 
ciintinuous qualitv improvement. 



I'hc rclaiivcK lovs number ot'Non survivors in the MSM prournm ( suggest caution should be used in 
interpreting this result. Nonetheless, the variances arc rclativelv close, suggesting that this finding will 
become more robust as more data become available, and eonsequentlv should not unduK worrv us. 
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